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The MAILING DATE of this communication appears on the cover sheet with the correspondence address « 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 
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earned patent term adjustment. See 37 CFR 1 .704(b). 
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1 )S Responsive to communication(s) filed on 15 March 2000 . 
2a)D This action is FINAL. 2b)IE This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 
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application from the International Bureau (PCT Rule 17.2(a)). 
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DETAILED ACTION 

1 . This action is responsive to communications: application filed on 03/16/2000, IDS filed 
on 12/17/2000. 

2. Claims 1-9 are pending in the case. Claims 1, 6, and 8 are independent claims. 

Information Disclosure Statement 

3. The listing of Gormish et al., US 5,337,362 in the specification is not a proper 
information disclosure statement. 37 CFR 1.98(b) requires a list of all patents, publications, or 
other information submitted for consideration by the Office, and MPEP § 609 A(l) states, "the 
list may not be incorporated into the specification but must be submitted in a separate paper." 
Therefore, unless the references have been cited by the examiner on form PTO-892, they have 
not been considered. 

Claim Rejections - 35 USC § 102 

4. The following is a quotation of the appropriate paragraphs of 35 U.S. C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

5. Claim 37 is rejected under 35 U.S.C. 102(b) as being anticipated by Covington et al., 
US 5,524,193 published 06/04/1996. 

Regarding independent claim 37, Covington discloses creating a document to be used 
with a multimedia annotation in col. 2 lines 39-41 and creating a multimedia annotation in col. 2 
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lines 49-50. Covington also discloses storing an image of the document and the multimedia 
annotation in col. 4 lines 8-1 1 and combining the document and the multimedia annotation to 
form a multimedia document in col. 2 lines 52-55. 

Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S. C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

7. Claims 1-4, 7-8, 13-16, 19-20, 25-28, 31-32, and 38-40 are rejected under 35 U.S.C 
103(a) as being unpatentable over Covington et aL, US 5,524,193 published 06/04/1996 in 
view of Gormish et aL, US 5,337,362 published 08/09/1994. 

Regarding independent claim 1, Covington teaches creating a multimedia annotation 
for a document and combining the document with the multimedia annotation to form a 
multimedia document in col. 2 lines 39-41 . What Covington does not teach is that the annotated 
document may be paper. Gormish does teach adding digital data into a paper document in col. 2 
lines 8-40 and col. 3 lines 7-25. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to combine Gormish into Covington to create the claimed invention. It would have 
been obvious and desirable to use the ability to place digitally encoded data onto a paper 
document taught by Gormish and use the said ability to implement the annotation method of 
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Covington so that the user could have had increased freedom to read the document in either 
paper form or electronic form. 

Regarding independent claim 13, Covington teaches creating a multimedia annotation 
for a document and combining the document with the multimedia annotation to form a 
multimedia document in col. 2 lines 39-41 . What Covington does not teach is that the annotated 
document may be paper. Gormish does teach adding digital data into a paper document in col. 2 
lines 8-40 and col. 3 lines 7-25. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to combine Gormish into Covington to create the claimed invention. It would have 
been obvious and desirable to use the ability to place digitally encoded data onto a paper 
document taught by Gormish and use the said ability to implement the annotation program on 
machine-readable medium of Covington so that the user could have had increased freedom to 
read the document in either paper form or electronic form. 

Regarding independent claim 25, Covington teaches a computer system comprising a 
data storage device, a processor coupled to the storage device, and inherently comprising a bus in 
col. 3 line 63 - col. 4 line 20. Covington also teaches creating a multimedia annotation for a 
document and combining the document with the multimedia annotation to form a multimedia 
document in col. 2 lines 39-41. What Covington does not teach is that the annotated document 
may be paper. Gormish does teach adding digital data, which could be annotations, into a paper 
document in col. 2 lines 8-40 and col. 3 lines 7-25. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to combine Gormish into Covington to create the claimed invention. It would have 



Application/Control Number: 09/526,03 1 Page 5 

Art Unit: 2176 

been obvious and desirable to use the ability to place digitally encoded data onto a paper 
document taught by Gormish and use the said ability to implement the annotation computer 
system of Covington so that the user could have had increased freedom to read the document in 
either paper form or electronic form. 

Regarding dependent claims 2, 14, and 26, which are dependent on claims 1,13, and 

25 respectively, Covington and Gormish teach the limitations of claims 1,13, and 25 as 
explained above. Covington does not teach a multimedia annotation represented as a bar code 
printed on a multimedia document. Gormish does teach a multimedia annotation represented as 
a bar code printed on a multimedia document in col. 1 lines 45-66 and also discloses a more 
advanced form of digital representation on paper in Fig. 2. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to combine Gormish into Covington to create the invention as claimed. It would have 
been obvious and desirable to use a bar code to represent the multimedia annotation on the 
multimedia document because bar code scanners were well known and inexpensive and would 
have made the invention economically attractive for interpreting the multimedia data. 

Regarding dependent claims 3, 15, and 27, which are dependent on claims 2, 14, and 

26 respectively, Covington and Gormish teach the limitations of claims 2, 14, and 26 as 
explained above. Covington teaches a multimedia annotation which encodes an audio sound in 
the abstract. 

Regarding dependent claims 4, 16, 28, and 40, which are dependent on claims 1, 13, 
25, and 38 respectively, Covington and Gormish teach the limitations of claims 1, 13, 25, and 38 
as explained above. Covington teaches a location indicator associated with the multimedia 
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annotation is placed on the multimedia document, wherein the location indicator indicates where 
the multimedia annotation can be retrieved and played in col. 2 lines 39-55. 

Regarding dependent claims 7, 19, and 31, which are dependent on claims 1,13, and 
25 respectively, Covington and Gormish teach the limitations of claims 1,13, and 25 as 
explained above. Covington does not teach that the multimedia document is a paper document. 
Gormish does teach adding digital data into a paper document which could be the combined 
multimedia document in col. 2 lines 8-40 and col. 3 lines 7-25. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to combine Gormish into Covington to create the claimed invention. It would have 
been obvious and desirable to use the paper document taught by Gormish and use said document 
to implement the multimedia document of Covington so that the user could have had increased 
freedom to read the document in either paper form or electronic form. 

Regarding dependent claims 8, 20, and 32, which are dependent on claims 1,13, and 
25 respectively, Covington and Gormish teach the limitations of claims 1,13, and 25 as 
explained above. Covington teaches combining the image of the paper document and the 
multimedia annotation to form a compound multimedia document in col. 2 lines 53-55. 
Covington also teaches storing the image of the document and multimedia annotation in col. 4 
lines 8-11. 

What Covington does not explicitly teach is generating an image of the paper document, 
but it was very well known to all of ordinary skill in the art at the time of the invention how to 
generate an image of a paper document. It would have been obvious to one of ordinary skill in 
the art at the time the invention was made to use the general knowledge of the art to modify 
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Covington such that paper multimedia documents could have been scanned into the computer 
system so that the user would have could have had increased freedom to read the document in 
either paper form or electronic form. 

Regarding dependent claim 38, which is dependent on claim 37, Covington teaches the 
limitations of claim 37. Covington teaches creating a multimedia annotation for a document and 
combining the document with the multimedia annotation to form a multimedia document in col. 
2 lines 39-41 . What Covington does not teach is that the annotated document may be paper. 
Gormish does teach adding digital data into a paper document in col. 2 lines 8-40 and col. 3 lines 
7-25. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to combine Gormish into Covington to create the claimed invention. It would have 
been obvious and desirable to use the ability to place digitally encoded data onto a paper 
document taught by Gormish and use the said ability to implement the annotation method of 
Covington so that the user could have had increased freedom to read the document in either 
paper form or electronic form. 

Regarding dependent claim 39, which is dependent on claim 38, Covington and 
Gormish teach the limitations of claim 38. Covington does not teach a multimedia annotation 
represented as a bar code printed on a multimedia document. Gormish does teach a multimedia 
annotation represented as a bar code printed on a multimedia document in col. 1 lines 45-66 and 
also discloses a more advanced form or digital representation on paper in Fig. 2. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to combine Gormish into Covington to create the invention as claimed. It would have 
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been obvious and desirable to use a bar code to represent the multimedia annotation on the 
multimedia document because bar code scanners were well known and inexpensive and would 
have made the invention economically attractive for interpreting the multimedia data. 

8. Claims 5-6, 9-10, 17-18, 21-22, 29-30, 33-34, and 42 are rejected under 35 
U.S.C. 103(a) as being unpatentable over Covington et al., US 5,524,193 published 
06/04/1996 in view of Gormish et al., US 5,337,362 published 08/09/1994 as applied to 
claims 1, 8, 13, 20, 25, and 32 above, and further in view of Stern, US 6,572,661 Bl filed 
01/11/1999. 

Regarding dependent claims 5, 17, 29, and 41, which are dependent on claims 4, 16, 
28, and 40 respectively, Covington and Gormish teach the limitations of claims 4, 16, 28, and 40 
as explained above. Covington does not teach a location indicator which is a Uniform Resource 
Locator. Stern does teach a location indicator which is a Uniform Resource Locator in col. 2 
lines 2-5 because the link points to an HTML file. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to combine Stern into Covington in view of Gormish to create the claimed invention. 
It would have been obvious and desirable to use a URL link as the location indicator because this 
would make the invention compatible with the internet, which would have been a great 
advantage resulting in extensive compatibility and ease of storing the multimedia files. 

Regarding dependent claims 6, 18, 30, and 42, which are dependent on claims 4, 16, 
28, and 40 respectively, Covington and Gormish teach the limitations of claims 4, 16, 28, and 40 
as explained above. Covington teaches a multimedia annotation which comprises an audio 
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sound, a video clip, and combination of both the audio sound and the video clip in col. 1 lines 
62-64 and col. 2 lines 39-41. 

Regarding dependent claims 9, 21, and 33, which are dependent on claims 8, 20, and 

32 respectively, Covington and Gormish teach the limitations of claims 8, 20, and 32 as 
explained above. Covington does not teach a location indicator which is a Uniform Resource 
Locator. Stern does teach a location indicator which is a Uniform Resource Locator in col. 2 
lines 2-5 because the link points to an HTML file. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to combine Stern into Covington in view of Gormish to create the claimed invention. 
It would have been obvious and desirable to use a URL link as the location indicator because this 
would make the invention compatible with the internet, which would have been a great 
advantage resulting in extensive compatibility and ease of storing the multimedia files. 

Regarding dependent claims 10, 22, and 34, which are dependent on claims 9, 21, and 

33 respectively, Covington, Gormish, and Stern teach the limitations of claims 9, 21, and 33 as 
explained above. Covington does not teach a location indicator represented as a bar code printed 
on a multimedia document. Gormish does teach a location indicator represented as a bar code 
printed on a multimedia document in col. 1 lines 45-66 and also discloses a more advanced form 
of digital representation on paper in Fig. 2. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to combine Gormish into Covington in view of Stem to create the invention as 
claimed. It would have been obvious and desirable to use a bar code to represent the location 
indicator on the multimedia document because bar code scanners were well known and 
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inexpensive and would have made the invention economically attractive for interpreting the 
multimedia data. 

9. Claims 11-12, 23-24, 35-36, and 43 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Covington et aL, US 5,524,193 published 06/04/1996 in view of Gormish 
et aL, US 5,337,362 published 08/09/1994 as applied to claim 8, 20, 32, and 38 above, and 
further in view of Halliday et aL, US 5,880,740. 

Regarding dependent claims 11, 23, and 35, which are dependent on claims 8, 20, and 
32 respectively, Covington and Gormish teach the limitations of claims 8, 20, and 32 as 
explained above. Covington does not teach sending a multimedia document to a recipient by 
electronic mail. Halliday does teach sending a multimedia document to a recipient by electronic 
mail in col 8 lines 5-28. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to combine Halliday into Covington in view of Gormish to create the claimed 
invention. It would have been obvious and desirable to send the electronic multimedia document 
to a recipient so that the user could share the information contained in the document with another 
person since that is the very purpose of documents and why they exist. 

Regarding dependent claims 12, 24, and 36, which are dependent on claims 1 1, 23, and 
35 respectively, Covington and Gormish teach the limitations of claims 1 1, 23, and 35 as 
explained above. Covington does not teach a recipient receiving an image of a paper document 
and a multimedia annotation in the form of a Multipurpose Internet Mail Extension (MIME). 
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Halliday does teach a recipient receiving an image of a paper document and a multimedia 
annotation in the form of a Multipurpose Internet Mail Extension (MIME) in col. 8 lines 5-28. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to combine Halliday into Covington in view of Gormish to create the claimed 
invention. It would have been obvious and desirable to send the electronic multimedia document 
to a recipient in the form of an electronic mail attachment so that the nature of the multimedia 
annotations be retained. One of ordinary skill in the art would have known to send multimedia 
files as attachments in electronic mail. 

Regarding dependent claim 43, which is dependent on claim 38, Covington and 
Gormish teach the limitations of claim 38. Covington does not teach sending the electronic 
multimedia document to a recipient, wherein the recipient receives the electronic multimedia 
document in the form of an attachment to an electronic mail. Halliday does teach sending the 
electronic multimedia document to a recipient, wherein the recipient receives the electronic 
multimedia document in the form of an attachment to an electronic mail in col. 8 lines 5-28. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to combine Halliday into Covington in view of Gormish to create the claimed 
invention. It would have been obvious and desirable to send the electronic multimedia document 
to a recipient in the form of an electronic mail attachment so that the nature of the multimedia 
annotations be retained. One of ordinary skill in the art would have known to send multimedia 
files as attachments in electronic mail. 



Conclusion 




* 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Peter J Smith whose telephone number is 703-305-593 1. The 
examiner can normally be reached on Mondays-Fridays 7:00am-3:30pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Joseph H Feild can be reached on 703-305-9792. The fax phone numbers for the 
organization where this application or proceeding is assigned are 703-746-7239 for regular 
communications and 703-746-7238 for After Final communications. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is 703-305-3900. 



PJS 

July 25, 2003 




